Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

filter_kubernetes: add owner references information into record. #9719

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

filter_kubernetes: add owner references information into record. #9719

wants to merge 1 commit into from
+168 −5

Conversation

shuaich
Copy link
Contributor

@shuaich shuaich commented Dec 12, 2024 •
edited
Loading

This change add owner references information into record when Owner_References parameter is set On.

This PR implements the feature request in #9631.

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
filters:
  - name:                kubernetes
          Match:                       kube_*
          ...
          Owner_References: On
  • Debug log output from testing the change
"kubernetes"=>{"pod_name"=>"fluentbit-gke-2p6b5", "namespace_name"=>"kube-system", "pod_id"=>"c759a5f5-xxxx-xxxx-9117-8a1dc0b1f907", "labels"=>{"component"=>"fluentbit-gke", "controller-revision-hash"=>"77665fff9", "k8s-app"=>"fluentbit-gke", "kubernetes.io/cluster-service"=>"true", "pod-template-generation"=>"1"}, "ownerReferences"=>[{"apiVersion"=>"apps/v1", "kind"=>"DaemonSet", "name"=>"fluentbit-gke", "uid"=>"1a12c3e2-d6c4-4a8a-b877-dd3c857d1aea", "controller"=>true, "blockOwnerDeletion"=>true}], "host"=>"xxx-2a9c049c-qgw3", "pod_ip"=>"10.128.0.111", "container_name"=>"fluentbit", "docker_id"=>"2acc16e490e8d6xxxacbb45fe6c519c4f55593e00cd59ff3178b3a92965c88", "container_hash"=>"xxx", "container_image"=>"sha256:5163d704420bb702ba4df39cc774c2961b4f5450ab843aa85f0e92aeb2321ea2"}}
  • Attached Valgrind output that shows no leaks or memory corruption was found
valgrind ./bin/flb-rt-filter_kubernetes
==1967769== Memcheck, a memory error detector
==1967769== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==1967769== Using Valgrind-3.20.0 and LibVEX; rerun with -h for copyright info
==1967769== Command: ./bin/flb-rt-filter_kubernetes
==1967769==
Test kube_core_base...                          [ OK ]
Test kube_core_no_meta...                       [ OK ]
Test kube_core_unescaping_text...               [ OK ]
Test kube_core_unescaping_json...               [ OK ]
Test kube_core_base_with_namespace_labels_and_annotations... [ OK ]
Test kube_core_base_with_owner_references...    [ OK ]
Test kube_options_use-kubelet_enabled_json...   [ OK ]
Test kube_options_use-kubelet_disabled_json...  [ OK ]
Test kube_options_merge_log_enabled_text...     [ OK ]
Test kube_options_merge_log_enabled_json...     [ OK ]
Test kube_options_merge_log_enabled_invalid_json... [ OK ]
Test kube_options_merge_log_disabled_json...    [ OK ]
Test kube_options_merge_log_trim_enabled_json... [ OK ]
Test kube_options_merge_log_trim_disabled_json... [ OK ]
Test kube_options_merge_log_key_json...         [ OK ]
Test kube_options_keep_log_enabled_json...      [ OK ]
Test kube_options_keep_log_disabled_json...     [ OK ]
Test kube_options_k8s_logging_parser_disabled_text_stdout... [ OK ]
Test kube_options_k8s_logging_parser_disabled_text_stderr... [ OK ]
Test kube_options_k8s_logging_exclude_disabled_text_stdout... [ OK ]
Test kube_options_k8s_logging_exclude_disabled_text_stderr... [ OK ]
Test kube_annotations_invalid_text...           [ OK ]
Test kube_annotations_parser_regex_with_time_text... [ OK ]
Test kube_annotations_parser_regex_with_time_invalid_text_1... [ OK ]
Test kube_annotations_parser_json_with_time_json... [ OK ]
Test kube_annotations_parser_json_with_time_invalid_json_1... [ OK ]
Test kube_annotations_parser_invalid_text_stdout... [ OK ]
Test kube_annotations_parser_invalid_text_stderr... [ OK ]
Test kube_annotations_parser_stdout_text_stdout... [ OK ]
Test kube_annotations_parser_stdout_text_stderr... [ OK ]
Test kube_annotations_parser_stderr_text_stdout... [ OK ]
Test kube_annotations_parser_stderr_text_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_1_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_1_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_2_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_2_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_3_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_3_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_4_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_4_stderr... [ OK ]
Test kube_annotations_parser_multiple_1_container_5_stdout... [ OK ]
Test kube_annotations_parser_multiple_1_container_5_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_1_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_1_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_2_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_2_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_3_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_3_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_4_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_4_stderr... [ OK ]
Test kube_annotations_parser_multiple_2_container_5_stdout... [ OK ]
Test kube_annotations_parser_multiple_2_container_5_stderr... [ OK ]
Test kube_annotations_exclude_default_text...   [ OK ]
Test kube_annotations_exclude_invalid_text_stdout... [ OK ]
Test kube_annotations_exclude_invalid_text_stderr... [ OK ]
Test kube_annotations_exclude_stdout_text_stdout... [ OK ]
Test kube_annotations_exclude_stdout_text_stderr... [ OK ]
Test kube_annotations_exclude_stderr_text_stdout... [ OK ]
Test kube_annotations_exclude_stderr_text_stderr... [ OK ]
Test kube_annotations_exclude_multiple_1_container_1_stdout... [ OK ]
Test kube_annotations_exclude_multiple_1_container_1_stderr... [ OK ]
Test kube_annotations_exclude_multiple_1_container_2_stdout... [ OK ]
Test kube_annotations_exclude_multiple_1_container_2_stderr... [ OK ]
Test kube_annotations_exclude_multiple_1_container_3_stdout... [ OK ]
Test kube_annotations_exclude_multiple_1_container_3_stderr... [ OK ]
Test kube_annotations_exclude_multiple_1_container_4_stdout... [ OK ]
Test kube_annotations_exclude_multiple_1_container_4_stderr... [ OK ]
Test kube_annotations_exclude_multiple_2_container_1_stdout... [ OK ]
Test kube_annotations_exclude_multiple_2_container_1_stderr... [ OK ]
Test kube_annotations_exclude_multiple_2_container_2_stdout... [ OK ]
Test kube_annotations_exclude_multiple_2_container_2_stderr... [ OK ]
Test kube_annotations_exclude_multiple_2_container_3_stdout... [ OK ]
Test kube_annotations_exclude_multiple_2_container_3_stderr... [ OK ]
Test kube_annotations_exclude_multiple_2_container_4_stdout... [ OK ]
Test kube_annotations_exclude_multiple_2_container_4_stderr... [ OK ]
Test kube_annotations_exclude_multiple_3_container_1_stdout... [ OK ]
Test kube_annotations_exclude_multiple_3_container_1_stderr... [ OK ]
Test kube_annotations_exclude_multiple_3_container_2_stdout... [ OK ]
Test kube_annotations_exclude_multiple_3_container_2_stderr... [ OK ]
Test kube_annotations_exclude_multiple_3_container_3_stdout... [ OK ]
Test kube_annotations_exclude_multiple_3_container_3_stderr... [ OK ]
Test kube_annotations_exclude_multiple_3_container_4_stdout... [ OK ]
Test kube_annotations_exclude_multiple_3_container_4_stderr... [ OK ]
Test kube_annotations_exclude_multiple_4_container_1_stdout... [ OK ]
Test kube_annotations_exclude_multiple_4_container_1_stderr... [ OK ]
Test kube_annotations_exclude_multiple_4_container_2_stdout... [ OK ]
Test kube_annotations_exclude_multiple_4_container_2_stderr... [ OK ]
Test kube_annotations_exclude_multiple_4_container_3_stdout... [ OK ]
Test kube_annotations_exclude_multiple_4_container_3_stderr... [ OK ]
Test kube_annotations_exclude_multiple_4_container_4_stdout... [ OK ]
Test kube_annotations_exclude_multiple_4_container_4_stderr... [ OK ]
SUCCESS: All unit tests have passed.
==1967769==
==1967769== HEAP SUMMARY:
==1967769==     in use at exit: 0 bytes in 0 blocks
==1967769==   total heap usage: 298,782 allocs, 298,782 frees, 106,224,699 bytes allocated
==1967769==
==1967769== All heap blocks were freed -- no leaks are possible
==1967769==
==1967769== For lists of detected and suppressed errors, rerun with: -s
==1967769== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@JeffLuoo
Copy link
Contributor

LGTM

@shuaich
Copy link
Contributor Author

shuaich commented Jan 8, 2025

Gentle ping. May I have the feedback on this PR?

edsiper
edsiper requested changes Jan 9, 2025
View reviewed changes
lib/cprofiles/src/cprof_profile.c Outdated
@@ -307,4 +307,4 @@ int cprof_profile_add_comment(struct cprof_profile *profile, int64_t comment)
profile->comments_count++;

return 0;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please do not touch this file

Copy link
Contributor Author

@shuaich shuaich Jan 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I had to manually apply local changes to cprof files to make the build and run the unit test. I then manually reverted those changes but still got the one line diff, which I don't still understand.

Do maintainers have the permission to revert this file change in this PR?

Copy link
Contributor Author

@shuaich shuaich Jan 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turns out there is an additional newline character appended by my vim editor. :set noeol fixes the one line diff. See https://stackoverflow.com/questions/1050640/how-to-stop-vim-from-adding-a-newline-at-end-of-file

@edsiper edsiper added this to the Fluent Bit v4.0.0 milestone Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edsiper edsiper edsiper requested changes

@leonardo-albertovich leonardo-albertovich Awaiting requested review from leonardo-albertovich leonardo-albertovich is a code owner

@fujimotos fujimotos Awaiting requested review from fujimotos fujimotos is a code owner

@koleini koleini Awaiting requested review from koleini koleini is a code owner

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
docs-required
Projects
None yet
Milestone
Fluent Bit v4.0.0
Development

Successfully merging this pull request may close these issues.
3 participants
@shuaich @JeffLuoo @edsiper